An ellipsoid constrained quadratic programming perspective to discriminative training of HMMs
نویسندگان
چکیده
In this paper, we reformulate the optimization in discriminative training (DT) of HMMs as an ellipsoid constrained quadratic programming (ECQP) problem, where a second order of the nonlinear space is adopted. We show that the unique optimal solution of ECQP can be obtained by an efficient line search and no relaxation is needed as in a general quadratically constrained quadratic programming (QCQP). Moreover, a subspace combination condition is introduced to further simplify it under certain cases. The concrete ECQP form of DT of HMMs is given based on a locality constraint and reasonable assumptions, and the algorithm can be conducted to update Gaussians jointly or separately in either sequential or batch mode. Under the perspective of ECQP, relationships between various popular DT optimization algorithms are discussed. Experimental results on two recognition tasks show that ECQP considerably outperforms other popular algorithms in terms of final recognition accuracy and convergence speed in iterations.
منابع مشابه
Covariance updates for discriminative training by constrained line search
We investigate the recent Constrained Line Search algorithm for discriminative training of HMMs and propose an alternative formula for variance update. We compare the method to standard techniques on a phone recognition task.
متن کاملRestructuring output layers of deep neural networks using minimum risk parameter clustering
This paper attempts to optimize a topology of hidden Markov models (HMMs) for automatic speech recognition. Current state-of-the-art acoustic models for ASR involve HMMs with deep neural network (DNN)-based emission density functions. Even though DNN parameters are typically trained by optimizing a discriminative criterion, topology optimization of HMMs is usually performed by optimizing a gene...
متن کاملDiscriminative training methods and their applications to handwriting recognition
This thesis aims to improve the performance of handwriting recognition systems by introducing the use of discriminative training methods. Discriminative training methods use data from all competing classes when training the recogniser for each class. We develop discriminative training methods for two popular classifiers: Hidden Markov Models (HMMs) and a prototype-based classifier. At the expen...
متن کاملQuadratically constrained quadratic programming for classification using particle swarms and applications
Particle swarm optimization is used in several combinatorial optimization problems. In this work, particle swarms are used to solve quadratic programming problems with quadratic constraints. The approach of particle swarms is an example for interior point methods in optimization as an iterative technique. This approach is novel and deals with classification problems without the use of a traditi...
متن کاملCombining Constraint Propagation and Discrete Ellipsoid-Based Search to Solve the Exact Quadratic Knapsack Problem
We propose an extension to the discrete ellipsoid-based search (DEBS) to solve the exact quadratic knapsack problem (EQKP), an important class of optimization problem with a number of practical applications. For the first time, our extension enables DEBS to solve convex quadratically constrained problems with linear constraints. We show that adding linear constraint propagation to DEBS results ...
متن کامل